RLBOA: A modular reinforcement learning framework for autonomous negotiating agents

Negotiation is a complex problem, in which the variety of settings and opponents that may be encountered prohibits the use of a single predefined negotiation strategy. Hence the agent should be able to learn such a strategy autonomously. To this end we propose RLBOA, a modular framework that facilitates the creation of autonomous negotiation agents using reinforcement learning. The framework allows for the creation of agents that are capable of negotiating effectively in many different scenarios. To be able to cope with the large size of the state and action spaces and diversity of settings, we leverage the modular BOA-framework. This decouples the negotiation strategy into a Bidding strategy, an Opponent model and an Acceptance condition. Furthermore, we map the multidimensional contract space onto the utility axis which enables a compact and generic state and action description. We demonstrate the value of the RLBOA framework by implementing an agent that uses tabular Q-learning on the compressed state and action space to learn a bidding strategy.We show that the resulting agent is able to learn well-performing bidding strategies in a range of negotiation settings and is able to generalize across opponents and domains

Postural change in volunteers: sympathetic tone determines microvascular response to cardiac preload and output increases

Purpose: Microvascular perfusion may be a non-invasive indicator of fluid responsiveness. We aimed to investigate which of the microvascular perfusion parameters truly reflects fluid responsiveness independent of sympathetic reflexes. Methods: Fifteen healthy volunteers underwent a postural change from head up tilt (HUT) to the supine position, diminishing sympathetic tone, followed by a 30° passive leg raising (PLR) with unaltered tone. Prior to and after the postural changes, stroke volume (SV) and cardiac output (CO) were measured, as well as sublingual microcirculatory perfusion (sidestream dark field imaging), skin perfusion, and oxygenation (laser Doppler flowmetry and reflectance spectroscopy). Results: In responders (subjects with >10 % increase in CO), the HUT to supine change increased CO, SV, and pulse pressure, while heart rate, systemic vascular resistance, and mean arterial pressure decreased. Additionally, microvascular flow index, laser Doppler flow, and microvascular hemoglobin oxygen saturation and concentration also increased. Conclusion: When preload and forward flow increase in association with a decrease in sympathetic activity, microvascular blood flow increases in the skin and in the sublingual area. When preload and forward flow increase with little to no change in sympathetic activity, only sublingual functional capillary density increases. Therefore, our results indicate that sublingual functional capillary density is the best pa

Tissue perfusion and oxygenation to monitor fluid responsiveness in critically ill, septic patients after initial resuscitation: a prospective observational study

Fluid therapy after initial resuscitation in critically ill, septic patients may lead to harmful overloading and should therefore be guided by indicators of an increase in stroke volume (SV), i.e. fluid responsiveness. Our objective was to investigate whether tissue perfusion and oxygenation are able to monitor fluid responsiveness, even after initial resuscitation. Thirty-five critically ill, septic patients underwent infusion of 250 mL of colloids, after initial fluid resuscitation. Prior to and after fluid infusion, SV, cardiac output sublingual microcirculatory perfusion (SDF: sidestream dark field imaging) and skin perfusion and oxygenation (laser Doppler flowmetry and reflectance spectroscopy) were measured. Fluid responsiveness was defined by a ≥5 or 10 % increase in SV upon fluids. In responders to fluids, SDF-derived microcirculatory and skin perfusion and oxygenation increased, but only the increase in cardiac output, mean arterial and pulse pressure, microvascular flow index and relative Hb concentration and oxygen saturation were able to monitor a SV increase. Our proof of principle study demonstrates that non-invasively assessed tissue perfusion and oxygenation is not inferior to invasive hemodynamic measurements in monitoring fluid responsiveness. However skin reflectance spectroscopy may be more helpful than sublingual SDF

Nitroglycerin reverts clinical manifestations of poor peripheral perfusion in patients with circulatory shock

Introduction: Recent clinical studies have shown a relationship between abnormalities in peripheral perfusion and unfavorable outcome in patients with circulatory shock. Nitroglycerin is effective in restoring alterations in microcirculatory blood flow. The aim of this study was to investigate whether nitroglycerin could correct the parameters of abnormal peripheral circulation in resuscitated circulatory shock patients.Methods: This interventional study recruited patients who had circulatory shock and who persisted with abnormal peripheral perfusion despite normalization of global hemodynamic parameters. Nitroglycerin started at 2 mg/hour and doubled stepwise (4, 8, and 16 mg/hour) each 15 minutes until an improvement in peripheral perfusion was observed. Peripheral circulation parameters included capillary refill time (CRT), skin-temperature gradient (Tskin-diff), perfusion index (PI), and tissue oxygen saturation (StO2) during a reactive hyperemia test (RincStO2). Measurements were performed before, at the maximum dose, and after cessation of nitroglycerin infusion. Data were analyzed by using linear model for repeated measurements and are presented as mean (standard error).Results: Of the 15 patients included, four patients (27%) responded with an initial nitroglycerin dose of 2 mg/hour. In all patients, nitroglycerin infusion resulted in significant changes in CRT, Tskin-diff, and PI toward normal at the maximum dose of nitroglycerin: from 9.4 (0.6) seconds to 4.8 (0.3) seconds (P <0.05), from 3.3°C (0.7°C) to 0.7°C (0.6°C) (P <0.05), and from [log] -0.5% (0.2%) to 0.7% (0.1%) (P <0.05), respectively. Similar changes in StO2 and RincStO2 were observed: from 75% (3.4%) to 84% (2.7%) (P <0.05) and 1.9%/second (0.08%/second) to 2.8%/second (0.05%/second) (P <0.05), respectively. The magnitude of changes in StO2 was more prono

Clinical assessment of peripheral perfusion to predict postoperative complications after major abdominal surgery early: A prospective observational study in adults

Introduction: Altered peripheral perfusion is strongly associated with poor outcome in critically ill patients. We wanted to determine whether repeated assessments of peripheral perfusion during the days following surgery could help to early identify patients that are more likely to develop postoperative complications.Methods: Haemodynamic measurements and peripheral perfusion parameters were collected one day prior to surgery, directly after surgery (D0) and on the first (D1), second (D2) and third (D3) postoperative days. Peripheral perfusion assessment consisted of capillary refill time (CRT), peripheral perfusion index (PPI) and forearm-to-fingertip skin temperature gradient (Tskin-diff). Generalized linear mixed models were used to predict severe complications within ten days after surgery based on Clavien-Dindo classification.Results: We prospectively followed 137 consecutive patients, from among whom 111 were included in the analysis. Severe complications were observed in 19 patients (17.0%). Postoperatively, peripheral perfusion parameters were significantly altered in patients who subsequently developed severe complications compared to those who did not, and these parameters persisted over time. CRT was altered at D0, and PPI and Tskin-diff were altered on D1 and D2, respectively. Among the different peripheral perfusion parameters, the diagnostic accuracy in predicting severe postoperative complications was highest for CRT on D2 (area under the receiver operating characteristic curve = 0.91 (95% confidence interval (CI) = 0.83 to 0.92)) with a sensitivity of 0.79 (95% CI = 0.54 to 0.94) and a specificity of 0.93 (95% CI = 0.86 to 0.97). Generalized mixed-model analysis demonstrated that abnormal peripheral perfusion on D2 and D3 was an independent predictor of severe postoperative complications (D2 odds ratio (OR) = 8.4, 95% CI = 2.7 to 25.9; D2 OR = 6.4, 95% CI = 2.1 to 19.6).Conclusions: In a group of patients assessed following major abdominal surgery, peripheral perfusion alterations were associated with the development of severe complications independently of systemic haemodynamics. Further research is needed to confirm these findings and to explore in more detail the effects of peripheral perfusion-targeted resuscitation following major abdominal surgery

Heterogeneously Catalyzed Continuous-Flow Hydrogenation Using Segmented Flow in Capillary Columns

Segmented flow in standard GC capillary columns, with a heterogeneous Pd catalyst on the walls, gave rapid information about catalytic processes in them. The residence time and conversion was monitored visually, greatly simplifying bench-scale optimization. Examples show the benefits of the elimination of pore diffusion and axial dispersion. Further, we demonstrated how to quickly identify deactivating species in multistep synthesis without intermediate workup

Trajectories of renal biomarkers and new-onset heart failure in the general population:Findings from the PREVEND study

AIMS: Renal dysfunction is one of the most critical risk factors for developing heart failure (HF). However, the association between repeated measures of renal function and incident HF remains unclear. Therefore, this study investigated the longitudinal trajectories of urinary albumin excretion (UAE) and serum creatinine and their association with new-onset HF and all-cause mortality.METHODS AND RESULTS: Using group-based trajectory analysis, we estimated trajectories of UAE and serum creatinine in 6881 participants from the Prevention of Renal and Vascular End-stage Disease (PREVEND) study and their association with new-onset HF and all-cause death during the 11-years of follow-up. Most participants had stable low UAE or serum creatinine. Participants with persistently higher UAE or serum creatinine were older, more often men, and more often had comorbidities, such as diabetes, a previous myocardial infarction or dyslipidaemia. Participants with persistently high UAE had a higher risk of new-onset HF or all-cause mortality, whereas stable serum creatinine trajectories showed a linear association for new-onset HF and no association with all-cause mortality.CONCLUSION: Our population-based study identified different but often stable longitudinal patterns of UAE and serum creatinine. Patients with persistently worse renal function, such as higher UAE or serum creatinine, were at a higher risk of HF or mortality.</p

HFrEF subphenotypes based on 4210 repeatedly measured circulating proteins are driven by different biological mechanisms

Background: HFrEF is a heterogenous condition with high mortality. We used serial assessments of 4210 circulating proteins to identify distinct novel protein-based HFrEF subphenotypes and to investigate underlying dynamic biological mechanisms. Herewith we aimed to gain pathophysiological insights and fuel opportunities for personalised treatment. Methods: In 382 patients, we performed trimonthly blood sampling during a median follow-up of 2.1 [IQR:1.1–2.6] years. We selected all baseline samples and two samples closest to the primary endpoint (PEP; composite of cardiovascular mortality, HF hospitalization, LVAD implantation, and heart transplantation) or censoring, and applied an aptamer-based multiplex proteomic approach. Using unsupervised machine learning methods, we derived clusters from 4210 repeatedly measured proteomic biomarkers. Sets of proteins that drove cluster allocation were analysed via an enrichment analysis. Differences in clinical characteristics and PEP occurrence were evaluated. Findings: We identified four subphenotypes with different protein profiles, prognosis and clinical characteristics, including age (median [IQR] for subphenotypes 1–4, respectively:70 [64, 76], 68 [60, 79], 57 [47, 65], 59 [56, 66]years), EF (30 [26, 36], 26 [20, 38], 26 [22, 32], 33 [28, 37]%), and chronic renal failure (45%, 65%, 36%, 37%). Subphenotype allocation was driven by subsets of proteins associated with various biological functions, such as oxidative stress, inflammation and extracellular matrix organisation. Clinical characteristics of the subphenotypes were aligned with these associations. Subphenotypes 2 and 3 had the worst prognosis compared to subphenotype 1 (adjHR (95%CI):3.43 (1.76–6.69), and 2.88 (1.37–6.03), respectively). Interpretation: Four circulating-protein based subphenotypes are present in HFrEF, which are driven by varying combinations of protein subsets, and have different clinical characteristics and prognosis. Clinical Trial Registration: ClinicalTrials.gov Identifier: NCT01851538 https://clinicaltrials.gov/ct2/show/NCT01851538. Funding: EU/ EFPIA IMI2JU BigData@Heart grant n° 116074, Jaap Schouten Foundation and Noordwest Academie.</p

Mucus Microbiome of Anastomotic Tissue During Surgery Has Predictive Value for Colorectal Anastomotic Leakage

OBJECTIVE: The aim of the present study is to investigate the association of gut microbiota, depending on treatment method, with the development of colorectal anastomotic leakage (AL). BACKGROUND: AL is a major cause for morbidity and mortality after colorectal surgery, but the mechanism behind this complication still is not fully understood. METHODS: Bacterial DNA was isolated from 123 "donuts" of patients where a stapled colorectal anastomosis was made and was analyzed using 16S MiSeq sequencing. In 63 patients, this anastomosis was covered with a C-seal, a bioresorbable sheath stapled to the anastomosis. RESULTS: In non-C-seal patients, AL development was associated with low microbial diversity (P = 0.002) and correspondingly with a high abundance of the dominant Bacteroidaceae and Lachnospiraceae families (P = 0.008 and 0.010, respectively). In C-seal samples, where AL rates were slightly higher (25% vs 17%), an association with the gut microbiota composition was almost undetectable. Only a few opportunistic pathogenic groups of low abundance were associated with AL in C-seal patients, in particular Prevotella oralis (P = 0.007). CONCLUSIONS: AL in patients without a C-seal can be linked to the intestinal microbiota, in particular with a low microbial diversity and a higher abundance of especially mucin-degrading members of the Bacteroidaceae and Lachnospiraceae families. In C-seal patients, however, it seems that any potential protective benefits or harmful consequences of the gut microbiota composition in regard to wound healing are negated, as progression to AL is independent of the initially dominant bacterial composition.This is an open access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal. http://creativecommons.org/licenses/by-nc-nd/4.0

Prognostic value of temporal patterns of global longitudinal strain in patients with chronic heart failure

BACKGROUND: We investigated whether repeatedly measured global longitudinal strain (GLS) has incremental prognostic value over repeatedly measured left ventricular ejection fraction (LVEF) and N-terminal pro B-type natriuretic peptide (NT-proBNP), and a single “baseline” GLS value, in chronic heart failure (HF) patients. METHODS: In this prospective observational study, echocardiography was performed in 173 clinically stable chronic HF patients every six months during follow up. During a median follow-up of 2.7 years, a median of 3 (25th–75th percentile:2–4) echocardiograms were obtained per patient. The endpoint was a composite of HF hospitalization, left ventricular assist device, heart transplantation, cardiovascular death. We compared hazard ratios (HRs) for the endpoint from Cox models (used to analyze the first available GLS measurements) with HRs from joint models (which links repeated measurements to the time-to-event data). RESULTS: Mean age was 58 ± 11 years, 76% were men, 81% were in New York Heart Association functional class I/II, and all had LVEF < 50% (mean ± SD: 27 ± 9%). The endpoint was reached by 53 patients. GLS was persistently decreased over time in patients with the endpoint. However, temporal GLS trajectories did not further diverge in patients with versus without the endpoint and remained stable during follow-up. Both single measurements and temporal trajectories of GLS were significantly associated with the endpoint [HR per SD change (95%CI): 2.15(1.34–3.46), 3.54 (2.01–6.20)]. In a multivariable model, repeatedly measured GLS maintained its prognostic value while repeatedly measured LVEF did not [HR per SD change (95%CI): GLS:4.38 (1.49–14.70), LVEF:1.14 (0.41–3.23)]. The association disappeared when correcting for repeatedly measured NT-proBNP. CONCLUSION: Temporal evolution of GLS was associated with adverse events, independent of LVEF but not independent of NT-proBNP. Since GLS showed decreased but stable values in patients with adverse prognosis, single measurements of GLS provide sufficient information for determining prognosis in clinical practice compared to repeated measurements, and temporal GLS patterns do not add prognostic information to NT-proBNP